Detection-free Bayesian Multi-object Tracking via Spatio-Temporal Video Bundles Grouping

This paper presents a conceptually simple but effective approach to track multi-object in videos without requiring elaborate supervision (i.e. training object detectors or templates offline). Our framework performs a bi-layer inference of spatio-temporal grouping to exploit rich appearance and motion information in the observed sequence. First, we generate a robust middle-level video representation based on clustered point tracks, namely video bundles. Each bundle encapsulates a chunk of point tracks satisfying both spatial proximity and temporal coherency. Taking the video bundles as vertices, we build a spatio-temporal graph that incorporates both the competitive and compatible relations among vertices. The multi-object tracking can be then phrased as a graph partition problem under the Bayesian framework, and we solve it by developing a robust belief propagation (RBP) algorithm. This algorithm improves the traditional belief propagation method by allowing a converged solution to be reconfigured during optimization, so that the inference can be re-activated once it gets stuck in local minima and thus conduct more reliable results. In the experiments, we demonstrate the superior performances of our approach on the challenging benchmarks compared with state-of-the-arts.